Skip to content

custom app integration possible? #424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
itsPreto opened this issue May 1, 2025 · 3 comments
Open

custom app integration possible? #424

itsPreto opened this issue May 1, 2025 · 3 comments
Labels
discussion For issues which are essentially discussions

Comments

@itsPreto
Copy link

itsPreto commented May 1, 2025

is there any way to access the fine-tuning features programmatically? I want to prepare the dataset and template and just call start_finetuning() or something

@dadmobile
Copy link
Member

Transformer Lab is made up of a React app front end (transformerlab-app) talking to a python engine backend (transformerlab-api) that does all of the work. So most things that the app does are possible to do programmatically via API.

If you are running Transformer Lab on your computer you can see the API docs by going to http://localhost:8338/docs .

e.g. In the case of fine-tuning, you need to first create a template for a fine-tune (doing this in the app might be easier). Then if you want to trigger a fine-tune you just need to create a new job with that template (you can do this over and over, possibly updating the template between calls).

If you're interested in doing this i would recommend joining out discord and somebody can maybe help you with any unclear details (the docs are quite lean).

We are also working on new workflow features that will make this more flexible...coming soon!

@deep1401 deep1401 added the discussion For issues which are essentially discussions label May 2, 2025
@deep1401
Copy link
Contributor

deep1401 commented May 8, 2025

Hi @itsPreto,
Its possible what we might have multiple ways to go about this including the ones Tony mentioned. Could you provide a more detailed explanation of the use case and I'll help you out!

@itsPreto
Copy link
Author

itsPreto commented May 8, 2025

Hey guys, first off I just want to extend my kudos because the project is sweet! I've used it several times for experiments on my m1 and love how accessible it is.

I’m working on an Ollama-based system that:
• Ingests archived chat data (from GPT, Claude, etc.)
• Transforms it into fine-tune-ready datasets based on user-defined goals

With goals to:
• Establish an easy workflow that takes users from their raw chat data to a fine tuned model-- ideally being guided by an LLM/Agent through the process.

Some example use cases would be:
• Personal therapist model that learns how to guide users through issues without enabling bad habits
• Dev assistant model fine-tuned my personal coding practices and patterns.

Essentially--
1. Raw chat logs → structured dataset
2. Apply template → trigger fine-tune via API
3. Return model/artifacts

Is there a recommended way to automate this using your backend? Also, curious how upcoming workflow features might support this.

Thanks for the great work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion For issues which are essentially discussions
Projects
None yet
Development

No branches or pull requests

3 participants